Skip to content

Add default proxied network isolation for registry benchmarks#96

Open
jdx wants to merge 2 commits intovltpkg:mainfrom
jdx:feat/vsr-network-isolation
Open

Add default proxied network isolation for registry benchmarks#96
jdx wants to merge 2 commits intovltpkg:mainfrom
jdx:feat/vsr-network-isolation

Conversation

@jdx
Copy link
Copy Markdown

@jdx jdx commented Apr 21, 2026

Summary

  • make registry-* benchmark runs default to a local toxiproxy-backed network path with a symmetric bandwidth cap, while keeping --network-profile=none as an escape hatch
  • route each proxied registry through its own loopback IP on the original upstream port so both metadata and tarball fetches stay inside the isolated path
  • expose the profile/rate controls in ./bench, document the behavior, and install/configure toxiproxy in setup and workflow paths so the default registry-bandwidth profile runs at 12000 KB/s in CI and locally

Benchmark Notes

  • Docker reproduction on next / registry-clean with npm only (--runs=3 --warmup=1) completed on main at 8.72s ± 0.17s.
  • The same benchmark on this branch, under the isolated registry-bandwidth profile, completed at 13.18s ± 0.09s.
  • Concise takeaway: the isolated path is slower, but it roughly halves run-to-run noise in this sample (±0.17s -> ±0.09s).

Verification

  • bash -n bench
  • bash -n scripts/network-isolation.sh
  • bash -n scripts/registry/common.sh
  • bash -n scripts/setup.sh
  • ./bench run --variation=registry-clean --fixtures=next --registries=vlt --dry-run
  • docker run --rm -v /tmp/vlt-benchmarks-main-96:/work -w /work vlt-benchmarks-local bash -lc '"'"'bash ./scripts/setup.sh >/tmp/setup.log && GH_REGISTRY="" ./bench run --fixtures=next --variation=registry-clean --registries=npm --runs=3 --warmup=1'"'"'
  • docker run --rm -v /tmp/vlt-benchmarks-pr-96:/work -w /work vlt-benchmarks-local bash -lc '"'"'bash ./scripts/setup.sh >/tmp/setup.log && GH_REGISTRY="" ./bench run --fixtures=next --variation=registry-clean --registries=npm --runs=3 --warmup=1'"'"'

@jdx
Copy link
Copy Markdown
Author

jdx commented Apr 22, 2026

gimme a bit to get around to reviewing this myself and seeing how it works, should get back to you tomorrow

@jdx jdx marked this pull request as ready for review April 22, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant